Image-based inventory control system

ABSTRACT

Inventory control systems for monitoring the removal and replacement of objects are described. An exemplary system includes at least one storage drawer, each storage drawer includes at least one storage location for storing objects, and configured to move in a first direction allowing increasing access to storage locations of the drawer, and a second direction allowing decreasing access to storage locations of the drawer. An image sensing device is provided to form at least one first image of the storage locations when a respective storage drawer moves in the second direction. The system includes a data processor configured to receive information representing images of the storage locations generated by the image sensing device, and determine an inventory condition of the objects stored in the respective storage drawer according to the at least one first image.

RELATED APPLICATIONS

This application claims the benefit of priority from provisional patentapplication No. 61/087,565, filed on Aug. 8, 2008 and entitledIMAGE-BASED INVENTORY CONTROL SYSTEM, the disclosure of which isincorporated herein in its entirety.

FIELD OF DISCLOSURE

The present disclosure relates to an inventory control system, morespecifically, to an image-based inventory control system for monitoringthe removal and replacement of objects, and identifying objects removedand returned to the system.

BACKGROUND AND SUMMARY OF THE DISCLOSURE

When tools are used in a manufacturing or service environment, it isimportant that tools be returned to a storage unit, such as a tool box,after use. Employers typically perform a manual inventory check of thetool box to minimize or eliminate the problem of misplacement or theftof expensive tools. Companies can conduct random audits of employee'stoolbox to prevent theft and monitor tool location.

Some industries have high standards for inventory control of tools, forpreventing incidents of leaving tools in the workplace environment wherethey could cause severe damages. For the aerospace industry, it isimportant to ensure that no tools are accidentally left behind in anaircraft or missile being manufactured, assembled or repaired. TheAerospace Industries Association even establishes a standard calledNational Aerospace Standard including recommended procedures, personnelmanagement and operations to reduce foreign object damage (FOD) toaerospace products. FOD is defined as any object not structurally partof the aircraft. The most common foreign objects found are nuts, bolts,safety wire, and hand tools. Inventory control over tools is critical toprevent tools from being left in an aircraft.

Some toolboxes try to build in inventory determination features to trackinventory conditions of tools stored in those toolboxes. For example,some toolboxes dispose contact sensors, magnetic sensors or infraredsensors in or next to each tool storage locations, to detect whether atool is placed in each tool storage location. Based on signals generatedby the sensors, the toolboxes are able to determine whether any toolsare missing. While this type of inventory check may be useful to someextents, it suffers from various drawbacks. For instance, if a sensordetects that something is occupying a storage location, the toolbox willdetermine that no tool is missing from that storage location. However,the toolbox does not know whether the right kind of tool is indeedplaced back in the toolbox or it is just some objects placed in thestorage location to cheat the system. Furthermore, disposing sensors fornumerous storage locations in a toolbox is tedious and costly, and thelarge number of sensors is prone to damages or malfunctions which willproduce false negative or positive alarms.

Accordingly, there is a need for an effective inventory control systemthat that could assist tracking and accounting for usage of tools andwhether they are properly put back after usage. There is also a need foran inventory control system which knows exactly what tool is removed orreturned to a tool box. Furthermore, as multiple workers may have accessto the same tool box, there is another need for an inventory controlsystem that can track a user and his or her usage of tools, to determineresponsibilities for any tool loss or misplacement.

This disclosure describes various embodiments of highly automatedinventory control systems that utilize uniquely timed machine vision andmethodology to capture images of a storage unit for identifying aninventory condition in the storage unit. Illustrative features mayinclude the ability to control access to the storage unit, determine ifobjects are in the storage unit or not, determine whether particulartools are in or missing from the storage unit, record images of storagelocations and/or their contents, save and retrieve images for auditpurposes, associate the images with a person who accessed the storageunit, and other features which will be described herein in detail.

According to one embodiment, an inventory control system for monitoringthe removal and replacement of objects, comprises at least one storagedrawer, each storage drawer including at least one storage location forstoring objects, and configured to move in a first direction allowingincreasing access to storage locations of the drawer, and a seconddirection allowing decreasing access to storage locations of the drawer.An image sensing device is provided to form at least one first image ofthe storage locations when a respective storage drawer moves in thesecond direction or after the respective storage drawer stops moving inthe second direction. The system includes a data processor configured toreceive information representing images of the storage locationsgenerated by the image sensing device, and determine an inventorycondition of the objects stored in the respective storage draweraccording to the at least one first image.

In one embodiment, the image sensing device is configured to form atleast one second image of the storage locations when the respectivestorage drawer moves in the first direction, before the respectivestorage drawer moves in the first direction, or after the respectivestorage drawer stops moving in the first direction. The data processoris configured to determine a change of the inventory condition of theobjects stored in the respective storage drawer according to adifference of the at least one first image and the at least one secondimage.

In one embodiment, an exemplary inventory control system includesmultiple storage drawers. Except for a bottom storage drawer, each ofthe multiple storage drawers is disposed on top of another storagedrawer. The image sensing device is disposed at a location sufficient toform an image of at least a portion of the storage locations of therespective storage drawer. The image sensing device may include a cameradisposed above the moving path of each respective storage drawer. Theimage sensing device may have a viewing field over a moving path of eachrespective storage drawer, and the image sensing device may be disposedat a location sufficient to form an image of at least a portion of therespective storage drawer when the portion of the storage drawer movespast the sensing field. In one embodiment, the image sensing device isconfigured to capture multiple partial images of the respective storagedrawer at different points in time when the storage drawer passes theviewing field. The data processor is configured to form a combined imageof the storage drawer by combining the multiple partial images of thestorage drawer captured at different points in time, and determines aninventory condition of the storage drawer based on the combined image.The system may further comprises a position detector configured todetect a position of each respective storage drawer when the storagedrawer moves. The data processor forms a combined image of the storagedrawer according to the multiple partial images and information ofpositions of the storage drawer detected by the position detector.

In still another embodiment, the image sensing device includes a cameraand a light redirecting device. The light redirecting device isconfigured to direct light reflected from each respective storage drawerto the camera. The camera is disposed at a location remote from thelight redirecting device. The light redirecting device may include oneor more minors, a prism or one or more fiber optic devices. The lightredirecting device may be disposed at a location above a moving path ofeach respective storage drawer when the storage drawer moves.

According to one embodiment, the image capturing device is configured toform an image of the respective storage drawer after detecting thestorage drawer stops from moving in the second direction. In one aspect,in forming the at least one first image, the image capturing deviceforms an image of storage locations of the storage drawer that are notaccessible by a user after detecting the storage drawer stops moving inthe second direction.

In one embodiment, the at least one first image is taken when thestorage drawer is moving. In another embodiment, the data processor isconfigured to associate the determined inventory condition with timeinformation indicating the time when the at least one first image wastaken. In still another embodiment, when a user is authenticated toaccess to a respective storage drawer, the image sensing device isconfigured to capture at least one second image of the storage locationsof the respective storage drawer before the respective storage drawermoves in the first direction; and the data processor is configured todetermine a change in the inventory condition of the objects stored inthe respective storage drawer according to a difference between the atleast one first image and the at least one second image.

In yet another embodiment, the image sensing device includes multiplecameras. Each camera is configured to capture a partial image of thestorage drawer. The data processor is configured to form a combinedimage of the storage drawer according to the partial image of thestorage drawer captured by each respective camera. In still anotherembodiment, the system further comprises an input device configured toreceive information submitted by a user accessing the system, and thedata processor is configured to authenticate the user based on thereceived information, and selectively grant or deny access to the useraccess to the system according to a result of the authenticating. In oneaspect, for each authenticated user who is granted access to the system,the data processor is configured to associate information related to theuser to an inventory condition associated with the granted access.

According to another embodiment, each storage location is configured tostore a pre-designated object, and the system has access to prestoredinformation identifying a relationship between each storage location andthe corresponding pre-designated object. In one aspect, the dataprocessor has access to a baseline image of the storage locations havingeach storage location occupied by the corresponding pre-designatedobject; and the data process is configured to determine the inventorycondition according to a difference between an image of the storagelocations and the baseline image. In another aspect, the data storagedevice stores a baseline image of the storage locations having eachstorage location unoccupied by the corresponding pre-designated object;and the data process is configured to determine the inventory conditionaccording to a difference between a captured image of the storagelocations and the baseline image.

According to still another embodiment, each respective storage locationis associated with an identifier. Depending on whether the storagelocation is being occupied by an object, the associated identifierappears in an image captured by the image sensing device in one of twodifferent manners. The data processor is configured to determine theinventory condition of objects by evaluating how each identifier appearsin the image captured by the image sending device. In one aspect, theidentifier is not viewable by the image sensing device when an object isstored in the respective storage location, and is viewable by the imagesensing device when an object is not stored in the respective storagelocation. In another aspect, the identifier is viewable by the imagesensing device when an object is stored in the respective storagelocation, and is not viewable by the image sensing device when an objectis not stored in the respective storage location. In still anotheraspect, each storage location is configured to store a pre-designatedobject. The data processor has access to information identifying a knownrelationship between each pre-designated object and each correspondingstorage location configured to store the pre-designated object. The dataprocessor is configured to determine an inventory condition of objectsby evaluating how each identifier appears in the image captured by theimage sensing device, and the relationship between each pre-designatedobject and each respective storage location. In a further aspect, theidentifier is unique to a corresponding object stored in the storagelocation. The data processor has access to information identifying eachobject stored in the system and information identifying a relationshipbetween each identifier and each corresponding object. The dataprocessor is configured to determine an inventory condition of theobjects by evaluating how each identifier appears in the image capturedby the image sensing device, the relationship identifying each objectstored in the system, and the information identifying the relationshipbetween each identifier and each corresponding object.

According to still another embodiment, each object includes an attachedidentifier unique to each object. The data processor has access toprestored information identifying each object stored in the system andinformation identifying a relationship between each pre-designatedobject and a respective identifier unique to each pre-designated object.The data processor is configured to determine an inventory condition ofobjects by evaluating the existence of at least one identifier in animage of the storage locations captured by the image sensing device, andthe relationship between each pre-designated object and a respectiveidentifier unique to each pre-designated object.

According to yet another embodiment, the data processor has access toinformation identifying all objects stored in the storage unit and imagecharacteristics unique to each object. The data processor is configuredto determine which objects are missing or which objects are in thesystem based on the captured image and the image characteristics uniqueto each object.

According to another embodiment, an exemplary inventory control systemfor monitoring the removal and replacement of objects, includes at leastone storage drawer, each storage drawer includes at least one storagelocation for storing objects, and each storage drawer is configured tomove between a first position, which allows no access to any storagelocations in the storage drawer, and a second position, which allowsaccess to at least one storage location in the storage drawer; an imagesensing device configured to form images of the storage drawers, whereinwhen or after a respective storage drawer moves from the second positionto the first position, the image sensing device is configured to form atleast one image of the storage locations of the respective storagedrawer; and a data processor configured to determine an inventorycondition of the respective storage drawer according to the at least oneimage of the storage locations formed after the respective storagedrawer moves from the second position to the first position. In oneaspect, the image sensing device includes a scan device configured toproject a scan light moving over the storage drawer and a cameraconfigured to receive the reflected scan light for forming images of thestorage drawer.

According to another embodiment, an inventory control system formonitoring the removal and replacement of objects, includes a storageunit including at least one storage location for storing objects; adetection device configured to detect a termination of each access tothe storage unit; an image sensing device is timed to capture at leastone first image of the storage locations after the detected terminationof access to the storage unit; a data processor configured to receiveinformation representing the at least one first image of the storagelocations, and determine an inventory condition of the objects stored inthe storage unit according to the at least one first image, associatethe determined inventory condition with the terminated access. In oneaspect, the image capturing device is timed to capture at least onesecond image of the storage locations after receipt of an access requestfor the access. The data processor is configured to determine a changein the inventory condition of the objects according to a differencebetween the at least one first image and the at least one second image.

According to another embodiment, an inventory control system formonitoring the removal and replacement of objects, comprises at leastone storage drawer, each storage drawer includes at least one storagelocation for storing objects, and each storage drawer is configured tomove between a first position, which allows no access to any storagelocations in the storage drawer, and a second position, which allowsaccess to at least one storage location in the storage drawer; and aimage sensing device configured to form images of the storage drawers,wherein when or after a respective storage drawer moves from the secondposition to the first position, the image sensing device is configuredto form at least one image of the storage locations of the respectivestorage drawer; and a data processor configured to determine aninventory condition of the respective storage drawer according to the atleast one image of the storage locations formed after the respectivestorage drawer moves from the second position to the first position.

In one aspect, the image sensing device includes a scan deviceconfigured to project a scan light moving over the storage drawer and acamera configured to receive the reflected scan light for forming imagesof the storage drawer.

Anther embodiment of this disclosure provides an inventory controlsystem for monitoring the removal and replacement of objects, comprisinga storage unit including at least one storage location for storingobjects, a detection device configured to detect a termination of eachaccess to the storage unit, an image sensing device timed to capture atleast one first image of the storage locations after the detectedtermination of access to the storage unit; and a data processorconfigured to receive information representing the at least one firstimage of the storage locations, and determine an inventory condition ofthe objects stored in the storage unit according to the at least onefirst image. In one aspect, the image capturing device is timed tocapture at least one second image of the storage locations after receiptof an access request for the terminated access; and the data processoris configured to determine a change in the inventory condition of theobjects according to a difference between the at least one first imageand the at least one second image. In another aspect, the system furtherincludes a lock device configured to selectively grant or prohibitaccess to the storage unit. The data processor is configured to collectinformation submitted by a user requesting access to the storage unit,authenticate the user based on the collected information, andselectively control the lock device to grant the user access to thestorage unit according to a result of the authenticating. For eachauthenticated user who is granted access to the storage unit, the dataprocessor is configured to associate information related to theauthenticated user to an inventory condition associated with the grantedaccess.

According to anther aspect, each storage location is configured to storea pre-designated object; and the data storage device stores informationidentifying a relationship between each storage location and thecorresponding pre-designated object. The data storage device may store abaseline image of the storage locations having each storage locationoccupied by the corresponding pre-designated object. The data process isconfigured to determine the inventory condition associated to eachaccess to the storage unit according to a difference between thecaptured image of the storage locations and the baseline image. Inanother aspect, the data storage device stores a baseline image of thestorage locations having each storage location unoccupied by thecorresponding pre-designated object; and the data process is configuredto determine the inventory condition associated to each access to thestorage unit according to a difference between the captured image of thestorage locations and the baseline image.

In another embodiment, each respective storage location is associatedwith an identifier. Depending on whether the storage location is beingoccupied by an object, the associated identifier appears in an imagecaptured by the image sensing device in one of two different manners.The data processor configured to determine the inventory condition ofobjects by evaluating how each identifier appears in the image capturedby the image sending device. According to one aspect, the identifier isnot viewable by the image sensing device when an object is stored in therespective storage location, and is viewable by the image sensing devicewhen an object is not stored in the respective storage location; or theidentifier is viewable by the image sensing device when an object isstored in the respective storage location, and is not viewable by theimage sensing device when an object is not stored in the respectivestorage location.

According to another aspect, each storage location is configured tostore a pre-designated object. The data processor has access toinformation identifying a known relationship between each pre-designatedobject and each corresponding storage location configured to store thepre-designated object, and the data processor is configured to determinean inventory condition of objects by evaluating how each identifierappears in the image captured by the image sensing device, and therelationship between each pre-designated object and each respectivestorage location.

According to another aspect, the identifier is unique to a correspondingobject stored in the storage location; the data processor has access toinformation identifying each object stored in the system and informationidentifying a relationship between each object and a respectiveidentifier unique to each object; and the data processor is configuredto determine an inventory condition of the objects by evaluating howeach identifier appears in the image captured by the image sensingdevice, the information identifying each object, and the informationidentifying the relationship between each object and a respectiveidentifier unique to each object.

According to still another aspect, each object includes an attachedidentifier unique to each object; the data processor has access toprestored information identifying each object stored in the system andinformation identifying a relationship between each pre-designatedobject and a respective identifier unique to each pre-designated object;and the data processor is configured to determine an inventory conditionof objects by evaluating the existence of at least one identifier in animage of the storage locations captured by the image sensing device, andthe relationship between each pre-designated object and a respectiveidentifier unique to each pre-designated object.

According a further aspect, the data processor has access to informationidentifying all objects stored in the storage unit and imagecharacteristics unique to each object; the data processor is configuredto determine the inventory condition of the objects based on thecaptured image and the image characteristics unique to each object.

According still another embodiment, each object includes an attachedidentifier unique to each object; the data storage device storesinformation identifying each object stored in the system and informationidentifying a relationship between each pre-designated object and arespective identifier unique to each pre-designated object; and the dataprocessor is configured to determine the inventory condition of objectsby evaluating the existence of each identifier, the informationidentifying each object, and the relationship between eachpre-designated object and a respective identifier unique to eachpre-designated object.

In another aspect, each storage location is configured to store apre-designated object; each respective storage location includes anidentifier unique to the pre-designated object stored in each respectivestorage location; the data storage device stores information identifyinga relationship between each pre-designated object and a respectiveidentifier unique to each pre-designated object; and the data processoris configured to determine whether any of the objects is missing fromthe storage locations according to the at least one image of the storagelocations captured after the detected access to the storage unit, andthe relationship between each pre-designated object and a respectiveidentifier unique to each pre-designated object.

According to a further aspect, the storage unit is configured to operatein a first mode allowing complete access to all storage locations, and asecond mode prohibiting access to any storage location; the imagesensing device is configured to capture a before-access image when thestorage unit operates in the first mode, and an after-access image,subsequent to the first image, when the storage unit operates in thesecond mode; and the data processor is configured to determine a changein an inventory condition of objects based on the before-access imageand the after-access image.

According to another embodiment, an exemplary system further includes acommunication device configured to form communication with dataprocessing system remote to the inventory control system, to transmitinventory information of the objects to the data processing system. Thedata processor may trigger an audio or visual warning signal when thedetermined inventory condition indicates that at least one object ismissing.

According to a further embodiment, a machine-readable medium carryinginstructions which, upon execution by a data processing system, controlthe data processing system to perform steps for managing a networkedinventory control system including at least one master storage systemassociated with multiple slave storage systems. The steps includesdetermining an authorization level of a user; responsive to thedetermined authorization level indicating that the user is authorized toaccess the master storage system, automatically granting the user accessto all the multiple slave storage systems associated with the materstorage system; and responsive to the determined authorization levelindicating that the user is authorized to access one of the slavestorage systems but not the master storage system, granting the useraccess to the authorized slave storage system, but not the masterstorage system.

According to another embodiment, a networked inventory control systemcomprises multiple storage systems. Each respective storage systemincludes an image sensing device configured to capture at least oneimage of objects stored in the storage system, and a data storage deviceconfigured to store the image captured by the image sensing device anduser information with respect to a user accessing the storage system. Adata processing system is coupled to the multiple storage systems via adata transmission network. The respective storage system is configuredto transmit stored image and user information to the data processingsystem for each access to the respective storage system. In one aspect,each storage system further includes a data processor configured todetermine an inventory condition of the objects stored in the storagesystem according to the image captured by the image sensing device, andtransmit information related to the inventory condition to the dataprocessing system. One of the storage systems may include multiplestorage drawers and access to each storage drawer is grantedindependently.

According a further embodiment, a network-enabled inventory controlsystem comprises a storage unit for storing objects, a data input deviceconfigured to receive user information from a respective user requestingaccess to the inventory control system, an image sensing deviceconfigured to capture at least one image of the storage locationscorresponding to each access to the inventory control system by therespective user, a data storage device configured to store the userinformation and the image captured by the image sensing device, acommunication device configured to form data communication with a remotedata processing system; and a data processor configured to controltransmission of data representing the image, the user information andthe corresponding access to the remote data processing system via thecommunication device. In one aspect, the storage unit includes multiplestorage drawers; and each storage drawer includes an access controldevice configured to selectively grant or deny access to the storagedrawer. The storage unit may include at least one storage drawer. Eachstorage drawer includes at least one storage location for storingobjects, and is configured to move in a first direction allowingincreasing access to storage locations of the drawer, and a seconddirection allowing decreasing access to storage locations of the drawer.An image sensing device is provided to form at least one first image ofthe storage locations when a respective storage drawer moves in thesecond direction or when the respective storage drawer stops moving inthe second direction. The data processor receives informationrepresenting images of the storage locations generated by the imagesensing device, and determines an inventory condition of the objectsstored in the respective storage drawer according to the at least onefirst image.

In one aspect, the exemplary system further includes an access controlsystem configured to selectively granting or denying access to thestorage unit. The communication device receives authorization data fromthe data processing system. The data processor is configured toauthenticate a user requesting access to the storage unit by determininga user authorization according to user information input by the user viathe data input device and the authorization data, and selectivelycontrol the access control device to grant access to the storage unitaccording to the determined user authorization.

The exemplary data processing systems as described herein may beimplemented using one or more computer systems and/or appropriatesoftware.

It is understood that embodiments, steps and/or features describedherein can be performed, utilized, implemented and/or practiced eitherindividually or in combination with one or more other steps, embodimentsand/or features.

Additional advantages and novel features of the present disclosure willbe set forth in part in the description which follows, and in part willbecome apparent to those skilled in the art upon examination of thefollowing, or may be learned by practice of the present disclosure. Theembodiments shown and described provide an illustration of the best modecontemplated for carrying out the present disclosure. The disclosure iscapable of modifications in various obvious respects, all withoutdeparting from the spirit and scope thereof. Accordingly, the drawingsand description are to be regarded as illustrative in nature, and not asrestrictive. The advantages of the present disclosure may be realizedand attained by means of the instrumentalities and combinationsparticularly pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is illustrated by way of example, and not by wayof limitation, in the accompanying drawings, wherein elements having thesame reference numeral designations represent like elements throughoutand wherein:

FIGS. 1 a and 1 b show exemplary storage units in which embodimentsaccording to this disclosure may be implemented;

FIG. 2 shows details inside an exemplary storage drawer operated in theopen mode;

FIG. 3 shows an exemplary tool storage system according to thisdisclosure;

FIGS. 4 a-4 c and 4 e are different views of the tool storage systemshown in FIG. 3;

FIG. 4 d illustrates how an exemplary image is stitched together;

FIGS. 5 a and 5 b are exemplary identifier designs for use in thisdisclosure;

FIGS. 6A, 6B, and 6C illustrate examples of timed image capturing;

FIGS. 7 a and 7 b are different views of another embodiment of cameradesigns;

FIG. 8 is a block diagram of an exemplary networked inventory controlsystem; and

FIG. 9 a-9 d are illustrative images of an exemplary audit record andimages taken during access to an exemplary system according to thisdisclosure.

DETAILED DESCRIPTIONS OF ILLUSTRATIVE EMBODIMENTS

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present disclosure. Specifically, operations ofillustrative embodiments that utilize machine vision to identifyinventory conditions of a storage unit are described in the context oftool management and tool inventory control. It will be apparent,however, to one skilled in the art that concepts of the disclosure maybe practiced or implemented without these specific details. Similarconcepts may be utilized in other types of inventory control systemssuch as warehouse management, jewelry inventory management, sensitive orcontrolled substance management, mini bar inventory management, drugmanagement, vault or security box management, etc. In other instances,well-known structures and devices are shown in block diagram form inorder to avoid unnecessarily obscuring the present disclosure.

Overview of Exemplary Tool Storage Systems

FIGS. 1 a and 1 b show exemplary storage units in which inventorycontrol systems according to this disclosure may be implemented. FIG. 1a is an exemplary tool storage system 100 including multiple storagedrawers 120. Each storage drawer 120 includes multiple storage locationsfor storing various types of tools. As used throughout this disclosure,a storage location is a location in a storage system for storing orsecuring objects. In one embodiment, each tool has a specificpre-designated storage location in the tool storage system.

Each storage drawer operates between a close mode, which allows noaccess to the contents of the drawer, and an open mode, which allowspartial or complete access to the contents of the drawer. When a storagedrawer moves from a close mode to an open mode, the storage drawerallows increasing access to its contents. On the other hand, if astorage moves from an open mode to a close mode, the storage drawerallows decreasing access to its contents. As shown in FIG. 1 a, allstorage drawers 120 are in close mode.

A locking device may be used to control access to the contents of thedrawers 120. Each individual drawer 120 may have its own lock ormultiple storage drawers 120 may share a common locking device. Onlyauthenticated or authorized users are able to access to the contents ofthe drawers.

The storage drawers may have different sizes, shapes, layouts andarrangements. FIG. 1 b shows another type of tool storage system 200which includes multiple storage shelves or compartments 220 and a singledoor 250 securing access to the storage shelves 250. The storage shelvesor compartments may come in different sizes, shapes, layouts andarrangements.

FIG. 2 shows details inside an exemplary storage drawer 120 in the openmode. Each storage drawer 120 includes a foam base 180 having at leastone storage location, such as cutouts 181, for storing tools. Eachcutout is specifically contoured and shaped for fittingly receiving atool with corresponding shapes. Tools may be secured in each storagelocation by using hooks, Velcro, latches, pressures from the foam, etc.

FIG. 3 shows an exemplary inventory control system implemented as a toolstorage system 300 according to this disclosure for storing tools.Storage system 300 includes a display 305, an access control device 306,such as a card reader, for verifying identity and authorization levelsof a user intending to access storage system 300, multiple tool storagedrawers 330 for storing tools. Tool storage system 300 includes an imagesensing device configured to capture images of contents or storagelocations of the system. The image sensing device may be lens-basedcameras, CCD cameras, CMOS cameras, video cameras, or any type of devicethat captures images. System 300 includes a data processing system, suchas a computer, for processing images captured by the image sensingdevice. Images captured or formed by the image sensing device areprocessed by the data processing system for determining an inventorycondition of the system or each storage drawer. The term inventorycondition as used throughout this disclosure means information relatingto an existence or non-existence condition of objects.

The data processing system may be part of tool storage system 300. Inone embodiment, the data processing system is a remote computer having adata link, such as a wired or wireless link, coupled to tool storagesystem 300; or a combination of a computer integrated in storage system300 and a computer remote to storage system 300. Detailed operations forforming images and determining inventory conditions will be discussedshortly.

Drawers 330 are similar to those drawers 120 shown in FIG. 1 a. Display305 is an input and/or output device of storage system 330, configuredto output information. Information entry via display 305 is possiblesuch as by using a touch screen display. Access control device 306 isused to limit access to tool storage drawers 330 to authorized usersonly. Access control device 306, through the use of one or more lockingdevices, keeps all storage drawers 330 locked in a closed position untilaccess control device 306 authenticates a user's authorization foraccessing storage system 300. Access control device 306 may use one ormore access authentication means to verify a user's authorizationlevels, such as by using a key pad to enter an access code, a keycardreader to read from a key card or fobs authorization level of a userholding the card or fob, biometric methods such as fingerprint readersor retinal scans, or other methods. If access control device 306determines that a user is authorized to access storage system 300, itunlocks some or all storage drawers 330, depending on the user'sauthorization level, allowing the user to remove or replace tools. Inone embodiment, access to each storage drawer 300 is controlled andgranted independently. Based on an assigned authorization or accesslevel, a user may be granted access to one or more drawers of system300, but not to other drawers. In one embodiment, access control device306 relocks a storage drawer 330 when or after a user closes the drawer.

The location of access control device 306 is not limited to the front ofstorage system 300. It could be disposed on the top of the system or ona side surface of the system. In one embodiment, access control device306 is integrated with display 305. User information for authenticationpurpose may be input through display device with touch screen functions,face detection cameras, fingerprint readers, retinal scanners or anyother types of devices used for verifying a user's authorization toaccess storage system 300.

FIGS. 4 a and 4 b show a partial perspective view of tool storage system300. As illustrated in FIG. 4 a, an access control device 306 in theform of a card reader is disposed on a side surface of the system.Storage system 300 includes an imaging compartment 330 which houses animage sensing device comprising three cameras 310 and a light directingdevice, such as a minor 312 having a reflection surface disposed atabout 45 degrees downwardly relative to a vertical surface, fordirecting light reflected from drawers 330 to cameras 310. The directedlight, after arriving to cameras 310, allow cameras 310 to form imagesof drawers 330. The shaded area 340 below minor 312 represents a viewingfield of the imaging sensing device of tool storage system 300. Mirror312 has a width equal to or larger than that of each storage drawer, andredirects the camera view downwards toward the drawers. FIG. 4 e is anillustrative side view of system 300 showing the relative positionbetween cameras 310, mirror 312 and drawers 330. Light L reflected fromany of drawers 330 to minor 312 is directed to cameras 310.

FIG. 4 b is a perspective view identical to FIG. 4 a except that a coverof imaging compartment 330 is removed to reveal details of the design.Each camera 310 is associated with a viewing field 311. As shown in FIG.4 b, the combined viewing fields of cameras 310 form the viewing field340 of the image sensing device. Viewing field 340 has a depth of x. Forinstance, the depth of viewing field 340 may be approximately 2 inches.

FIG. 4 c is an alternative perspective view of tool storage system 300shown in FIG. 4 a, except that a storage drawer 336 now operates in anopen mode allowing partial access to its contents or storage locationsin storage drawer 336.

This arrangement of cameras 310 and mirror 312 in FIGS. 4 a-4 c allowscameras 310 the capability of capturing images from the top drawer tothe bottom drawer, without the need to substantially change its focallength.

In one embodiment, cameras 310 capture multiple partial images of eachstorage drawer as it is opened or closed. Each image captured by cameras310 may be associated with a unique ID or a time stamp indicating thetime when the image was captured. Acquisition of the images iscontrolled by a data processor in tool storage system 300. In oneembodiment, the captured images are the full width of the drawer butonly approximately 2 inches in depth. The captured images overlapsomewhat in depth and/or in width. As shown in FIG. 4D, the partialimages 41-45 taken by different cameras 310 at different points in timemay be stitched together to form a single image of partial or entiredrawer and its contents and/or storage locations. This stitching may beperformed by the data processor or by an attached or remote computerusing off-the-shelf software programs. Since images are captured inapproximately two inch slices multiple image slices are captured by eachcamera. One or more visible scales may be included in each drawer. Theprocessor may monitor the portion of the image that contains the scalein a fast imaging mode similar to video monitoring. When the scalereaches a specified or calculated position, the data processing systemcontrols the image sensing device to capture and record an image slice.The scale may also assist in photo stitching. Additionally a patternsuch as a grid may be applied to the surface the drawer. The patterncould be used to assist the stitching or image capture process.

In another embodiment, the image sensing device includes larger mirrorsand cameras with wide angle lens, in order to create a deeper view fieldx, such that the need for image stitching can be reduced or entirelyeliminated.

In one embodiment, one or more line scan cameras are used to implementthe image sensing device. A line scan camera captures an image inessentially one dimension. The image will have a significant widthdepending on the sensor, but the depth is only one pixel. A line scancamera captures an image stripe the width of the tool drawer but onlyone pixel deep. Every time drawer 330 moves by a predetermined partialamount the camera will capture another image stripe. In this case theimage stripes must be stitched together to create a usable full drawerimage. This is the same process used in many copy machines to capture animage of the document. The document moves across a line scan camera andthe multiple image stripes are stitched together to create an image ofthe entire document.

In addition to a minor, it is understood that other devices, such asprisms, a combination of different types of lens including flat,concave, and/or convex, fiber optics, or any devices may direct lightfrom one point to another may be used to implement the light directingdevice for directing light coming from an object to a remote camera.Another option could be the use of fiber optics. The use of lightdirecting device may introduce distortions into the captured images.Calibrations or image processing may be performed to eliminate thedistortions. For instance, cameras 310 may first view a known simplegrid pattern reflected by the light directing device and create adistortion map for use by the data process processor to adjust thecaptured image to compensate for mirror distortion.

For better image capture and processing, it may be desirable tocalibrate the cameras. The cameras may include certain build variationswith respect to image distortion or focal length. The cameras can becalibrated to reduce distortion in a manner similar to how the minordistortion can be reduced. In fact, the minor calibration couldcompensate for both camera and minor distortion, and it may be the onlydistortion correction used. Further, each individual cameras may becalibrated using a special fixture to determine the actual focal lengthof their lenses, and software can be used to compensate for thedifferences from camera to camera in a single system.

In one embodiment, the image sensing device does not include any mirror.Rather, one or more cameras are disposed at the location where mirror312 was disposed. In this case, the cameras point directly down atstorage drawers 330 when then move. In another embodiment, each storagedrawer 330 has one or more associated cameras for capturing images forthat storage drawer.

Determination of Inventory Conditions

System 300 determines the presence or absence of tools in drawers 330based on captured images using a variety of possible strategies.Suitable software may be executed by the embedded processor or anattached computer (PC) for performing inventory determinations based oncaptured images.

In one example, system 300 determines an inventory condition of astorage drawer based on empty locations in the drawer. Each storagelocation in the drawer is configured to store a pre-designated object,such as a pre-designated tool. A non-volatile memory device of system300 stores information identifying a relationship between each knownstorage location in the drawer and its corresponding pre-designatedobject. The memory device also stores information of a baseline image ofthe drawer having each of its storage locations occupied by thecorresponding pre-designated object. In determining an inventorycondition of the drawer, the data processor compares an image of thedrawer and the baseline image. Based on a difference of the images, thedata processor determines which storage location in the drawer is notoccupied by its corresponding pre-designated object. The identify of themissing object is determined based on the stored relationshipidentifying each storage locations and their correspondingpre-designated objects.

In another embodiment, the baseline image includes an image with allstorage locations in a drawer unoccupied by their correspondingpre-designated objects. Based on a comparison of the captured image andthe baseline image, system 300 determines which storage location havecorresponding pre-designated objects and which locations do not. Missingobjects may be determined based on the stored relationship identifyingeach storage locations and their corresponding pre-designated objects.

In still another embodiment, the baseline image has some storagelocations occupied by their respective pre-designated objects, whileother locations are not occupied by their respective pre-designatedobjects. A non-volatile memory device of system 300 stores informationidentifying a relationship between each known storage location in thedrawer and its corresponding pre-designated object, and information onwhich locations are or are not occupied by their correspondingpre-designated objects. The system compares a captured image of a drawerwith the baseline image, and determines a difference. Based on thedetermined difference and the information on which locations are or arenot occupied by their corresponding pre-designated objects, system 300determines which storage location have corresponding pre-designatedobjects and which locations do not. Missing objects may be determinedbased on the stored relationship identifying each storage locations andtheir corresponding pre-designated objects.

Another embodiment according to this disclosure utilizes speciallydesigned identifier for determining an inventory condition of objects.Depending on whether a storage location is being occupied by an object,an associated identifier appears in one of two different manners in animage captured by the image sensing device. For instance, an identifiermay appear in a first color when the associated storage location isoccupied by a tool and a second color when the associated storagelocation is unoccupied. The identifiers may be texts, one-dimensional ortwo-dimensional bard code, patterns, dots, code, symbols, figures,numbers, LEDs, lights, flags, etc., or any combinations thereof. Thedifferent manners that an identifier may appear in an image captured bythe image sensing device include images with different patterns,intensities, forms, shapes, colors, etc. Based on how each identifierappears in a captured image, the data processor determines an inventorycondition of the object.

FIG. 5 shows an embodiment of identifier designs. As shown in FIG. 5,storage location 51 is designated to store tool 510, and storagelocation 52 is currently occupied by its designated tool 520. Storagelocation 53 is not occupied by its designated tool. Each storagelocation 51, 52, 53 has an associated identifier. Depending on whethereach storage location 51-53 is being occupied by a corresponding tool,each identifier appears in an image captured by cameras 310 in one oftwo different manners. For example, each identifier may not be viewableby cameras 310 when a corresponding tool is stored in the respectivestorage location, and becomes viewable by cameras 310 when an object isnot stored in the respective storage location. Similarly, a differentembodiment may have an identifier viewable by the image sensing devicewhen an object is stored in the respective storage location, and is notviewable by the image sensing device when an object is not stored in therespective storage location.

For instance, the bottom of storage locations 51-53 includes anidentifier made of retro-reflective material. Since storage locations 51and 53 are not occupied by their respective designated tools, theirassociated identifiers 511 and 513 are viewable to the image sensingdevice. On the other hand, storage location 52 is currently occupied byits designated tool, its identifier is blocked from the view of theimage sensing device. When the particular tool is stored in the storagelocation, the identifier is blocked from the view of the image sensingdevice and not viewable by the image sensing device. On the other hand,if the storage location is not occupied by the particular tool, theidentifier is viewable by the image sensing device and shows up as ahigh intensity area on an image of the drawer. Accordingly, a highintensity area represents a missing tool. The system 300 detectslocations with missing tools and correlates the empty locations withstored relationship identifying each storage locations and theircorresponding tools. The system 300 determines which tools are not intheir specified locations in a drawer. It is understood that theidentifiers may be implemented in many different manners. For instance,the identifiers may be designed to create a high intensity image when astorage location is occupied and an image with less intensity when thestorage location is occupied.

In one embodiment, each identifier is implemented with a contact sensorand an LED. As shown in FIG. 5 b, storage location 61 is associated witha contact sensor 62 and an LED 63. When contact sensor 61 senses a toolis in storage location 61, a signal is generated by contact sensor 61and controls to turn off power supply to LED 63. On the other hand, ifcontact sensor 62 detects that a tool is not in storage location 61,control sensor 62 generates a control signal which controls to turn onLED 63, which creates a high intensity area in an image captured by theimage sensing device. Each high intensity area in an image indicates astorage location without an associated tool. The system 300 identifiesremoved or missing tools by determining which storage locations are notoccupied by tools and pre-stored information identifying correspondingtools of the locations. In still another embodiment, the identifier isunique to the pre-designated tool stored in each respective storagelocation. The data processor is configured to determine an inventorycondition by evaluating whether at least one viewable identifier existsin an image of the storage locations captured by the image sensingdevice, and pre-stored relationship between each pre-designated objectand a respective identifier unique to each pre-designated object.

In still another embodiment, an identifier associated with a storagelocation creates a high intensity image when the storage location isoccupied, and a lower intensity image when the storage location isunoccupied. The system 300 determines which tools exist based ondetected identifiers and pre-stored information identifying arelationship between each storage location and the correspondingpre-designated object. In another embodiment, the identifier is uniqueto a pre-designated object stored in each respective storage location.The system 300 determines an inventory condition of existing objects byevaluating identifiers that exist in an image of the storage locationscaptured by the image sensing device, and pre-stored relationshipbetween each pre-designated object and a respective identifier unique toeach pre-designated object.

In still another embodiment, each object stored in the system 300includes an attached identifier unique to each object. The dataprocessor has access to prestored information identifying each toolstored in the system and known information identifying a relationshipbetween each object and a respective identifier unique to eachpre-designated object. The data processor determines an inventorycondition of objects by evaluating identifiers that exist in an image ofthe storage locations captured by the image sensing device, and therelationship between each pre-designated object and a respectiveidentifier unique to each pre-designated object. For instance, system300 stores a list of tools stored in the system and their correspondingunique identifiers. After cameras 310 captures an image of a storagedrawer, the data processor determines which identifier or identifiersare in the image. By comparing the identifiers appearing in the imagewith list of tools and their corresponding unique identifiers, the dataprocessor determines which tools are in the system and which ones arenot.

As discussed earlier, identifiers associated with the storage locationsmay be used to determine which locations have missing objects. Accordingto one embodiment, system 300 does not need to know the relationshipbetween each storage location and the corresponding object. Rather, eachidentifier is unique to a corresponding object stored in the storagelocation. The data processor of system 300 has access to pre-storedinformation identifying a relationship between each identifier and thecorresponding object, and information identifying each object. In otherwords, system 300 has access to an inventory list of every object storedin system 300 and its respective unique identifier. When an empty toolstorage location is detected by system 300, the corresponding identifieris extracted from the image and decoded by system software. As eachidentifier is unique to a corresponding object, system 300 is able todetermine which object is missing by checking the relationship betweeneach identifier and the corresponding object, and the inventory list ofobjects. Each identifier unique to an object stored in a storagelocation may be disposed next to the storage location or in the storagelocation. In one embodiment, the identifier is disposed next to thestorage location and is always viewable to the image sensing device nomatter whether the location is occupied by an object or not. In anotherembodiment, when an identifier is disposed in the correspondinglocation, the identifier is not viewable to the image sensing devicewhen the location is occupied by an object, and is viewable to the imagesensing device when the location is not occupied by an object.

An embodiment of this disclosure utilizes combinations of baselineimages and identifiers unique to objects to determine an inventorystatus. For example, a baseline image may include information of astorage drawer with all storage locations occupied with their respectivecorresponding objects, wherein each storage location is associated withan identifier unique to an object stored in the storage location. Aninventory condition is determined by comparing an image of the storagelocations and the baseline image, to determine which locations areoccupied by objects and/or which locations have missing objects.Identifications of the missing objects are determined by identifying theidentifier associated with each storage location with missing object.

Another embodiment of this disclosure utilizes unique combinations ofidentifiers to determine an inventory status. For instance, each storagelocation may have a first type of identifier disposed in the locationand a second type of identifier unique to an object stored in thestorage location and disposed next to the storage location. The firsttype of identifier is viewable to an image sensing device when thelocation is not occupied by an object and not viewable by an imagesensing device when the location is occupied by an object. The firsttype of identifier may be made of retro-reflective material. If astorage location is not occupied by an object corresponding to thestorage location, the identifier of the first type is viewable by theimage sensing device and shows up as a high intensity area. Accordingly,each high intensity area represents a missing object, which allowssystem 300 to determine which locations having missing objects. Based onidentifiers of the second type associated with those locations withmissing objects, system 300 identifies which objects are missing fromsystem 300. Consequently, an inventory condition of system 300 isdetermined.

According to still another embodiment, system 300 uses image recognitionmethods to identify an object missing from system 300. System 300 hasaccess to an inventory list indicating which tools are stored in eachdrawer or system 300. However, system 300 does not have to know wherethe tools are stored. The tools are placed in foam cutout locationsspecific for each tool. Using characteristics such as size, shape,color, and other parameters image recognition software identifies eachtool in the drawer. Missing tools are simply the tools on the inventorylist that are not identified as being in the drawer.

System 300 records access information related to each access. The accessinformation includes time, user information related to the access,duration, user images, images of storage locations, identities ofstorage units or contents of the storage system, objects in the storagesystem, etc., or any combinations thereof. In one embodiment, system 300includes a user camera that captures and stores image of the personaccessing storage system 300 each time access is authorized. For eachaccess by a user, system 300 determines an inventory condition andgenerates a report including associating the determined inventorycondition with access information.

Timed Image Capturing

Embodiments of this disclosure utilize uniquely timed machine imaging tocapture images of system 300 and determine an inventory condition ofsystem 300 according to the captured images. In one embodiment, system300 activates or times imaging of a storage drawer based on drawerlocations and/or movements, in order to create efficient and effectiveimages. For instance, a data processor of the system 300 uses drawerpositions to determine when to take overlapping partial images asdiscussed relative to FIGS. 4 a-4 e, to assure full coverage of a drawerbeing accessed by a user. In another example, drawer positioninformation may be useful to the stitching software in the constructionof a full drawer image. Drawer position information may be used to helplocate the positions of the cutouts in the drawer.

In one embodiment, the data processor of system 300 controls the imagesensing device to form images of a drawer based on a pre-specifiedmanner of movement by the drawer. For instance, for each access, system300 only takes images of the drawer when it is moving in a specifiedmanner or in a predetermined direction. According to one embodiment, theimage sensing device takes images when a drawer is moving in a directionthat allows decreasing access to its contents or after the drawer stopsmoving in the direction allowing decreasing access to its contents. Forexample, cameras may be controlled to take pictures of drawers when auser is closing a drawer, when or after a drawer stops moving in aclosing direction or when the drawer is completely closed. In oneembodiment, no images are taken when the drawer is moving in a directionallowing increasing access to its contents, such as when a drawer movesfrom a close position to an open position.

FIG. 6 shows an operation of this embodiment in the setting of anexemplary system described in FIGS. 4 a-4 d. As shown in FIG. 6 a, auser partially opens drawer 330 to expose storage locations in shadedarea 331. Since the user only opens drawer 330 half way, the user has noaccess to storage locations in area 336. After the user finds the toolhe needs from area 331, the user starts to close drawer 330 (FIG. 6 b).When sensors in system 300 detect the closing movement of drawer 330,which allows decreasing access to contents, the data processor activatesthe image sensing device, such as cameras 310, to capture partial imagesof shaded area 331 until drawer 330 is fully closed (FIG. 6 c). Sincethe user never has any access to area 336, it is safe to assume that aninventory condition relative to area 336 remains unchanged from theprevious access. However, for area 331, since the user had access tothat area, an inventory associated with that area needs to be updated.Any changes in access or replacement of tools would occur only in area331. Therefore, system 300 determines an inventory condition of drawer330 associated with the access by the user based on the captured imagecovering area 331 and inventory information related to area 336 of aprevious access, the information of which may be retrieved from anon-volatile memory device of system 300 that stores inventoryinformation associated with each access to system 300. The determinedinventory condition for drawer 330 is then stored in the non-volatilememory device. In one embodiment, the non-volatile memory device storesan initial inventory condition of drawer 300 which represents a baselineinventory condition with which later inventory conditions may compare.For instance, after each auditing of tool inventory condition, system300 stores the inventory condition after the audit as the baselineinventory condition.

Locations, movements and moving directions of each storage drawer may bedetermined by using sensors to measure location or movement sensorsrelative to time. For instance, location information relative to twopoints in time may be used to derive a vector indicating a movingdirection.

Examples of sensors for detecting a position, movement or movingdirection of storage drawers include a sensor or encoder attached to adrawer to detect its position relative to the frame of system 300; anon-contact distance measuring sensor for determining drawer movementrelative to some position on the frame of the system 300, such as theback of the system 300; etc. Non-contact sensors may include optical orultrasonic sensors. A visible scale or indicator viewable by cameras 310may be included in each drawer, such that camera 210 could read thescale to determine drawer position.

A change in an inventory condition, such as removal of tools, occurringin the current access may be determined by comparing inventoryconditions of the current access and the access immediately before thecurrent access. If one or more objects are missing, system 300 maygenerate a warning signal, such as audible or visual, to the user,generate a notice to a remote server coupled to system 300, etc.

In another embodiment, the image sensing device is configured to formimages of the storage locations both when storage drawer 330 moves in adirection allowing increasing access to its contents, and when storagedrawer 330 subsequently moves in a direction allowing decreasing accessto its contents. For example, when a user opens drawer 330 to retrievetools, the moving direction of drawer 330 triggers cameras 310 tocapture images of drawer contents when it moves. The captured image maybe designated as a “before access” image representing a status before auser has accessed the contents of each storage drawer. An inventorycondition is determined based on the captured images. This inventorycondition is considered as a “before access” inventory condition.Cameras 310 stops capturing images when drawer 330 stops moving. Whenthe user closes drawer 330, the moving direction of drawer 330 triggerscameras 310 to capture images of drawers 330 again until it stops orreaches a close position. An inventory condition of the drawer isdetermined based on images captured when the user closes drawer 330.This determined inventory condition is designated as an “after access”inventory condition. A difference between the before access inventorycondition and the after access inventory condition indicates a removalor replacement of tools. Other embodiments of this disclosure controlcameras to take the “before access” image before a storage drawer isopened or after the storage drawer is fully opened or when its contentsare accessible to a user. According to another embodiment, the imagesensing device is timed to take an image of each drawer 330 when it wasdetected that access by a user is terminated. As used herein in thisdisclosure, terminated access is defined as a user no longer havingaccess to any storage locations, such as when drawer 330 is closed orlocked, when door 250 is closed or locked, etc., or an indication by theuser or the system that access to the storage system is no longerdesired, such as when a user signs off, when a predetermined period oftime has elapsed after inactivity, when a locking device is locked by auser or by system 300, etc. For each access, a position detector orcontact sensor is used to determine whether drawer 330 is closed. Afterthe drawer is closed, the image sensing device captures an image ofdrawer 330. The data processing system then determines an inventorycondition based on the captured image or images. A difference in theinventory condition may be determined by comparing the determinedinventory condition of the current access to that of the previousaccess.

FIGS. 7 a and 7 b show an exemplary drawer having cameras configured tocapture images of the drawer when it is closed. FIG. 7 a is a top viewof a drawer 330 having three cameras 770. Cameras 770 have sufficientwidths of viewing fields to cover the entire width of drawer 330. FIG. 7b is a side view of drawer 330 shown in FIG. 7 a. Camera 710 tilts downa specific angle and has a sufficiently large viewing field to cover theentire length L of drawer 330. In one embodiment, cameras 710 does nothave to cover the entire length L with one image. Rather, camera 710 maybe rotatably attached to a hinge 711, which allows camera to tilt up anddown vertically, to cover different sections of drawer 330. Imagescaptured by cameras 710 are stitched or combined to form an image of theentire drawer.

It is understood that other camera configurations or designs may beutilized to capture images of drawer 330 when it is closed. In oneembodiment, one or more moving cameras are used to capture images of adrawer after it is closed. In this embodiment, the cameras areconfigured to move over the drawer and capture image slices that can bestitched together to create a full drawer images. The cameras may bemoved by a motor along a rail. Either 2D or line scan cameras can beused in this model. A sensor may be used to determine the location ofthe cameras to assist in stitching or other functions such as cameraposition control. A variation of this model uses a stationary camera foreach drawer viewing across the top of the drawer and a 45 degree movingminor that moves over the draw and redirects the camera view towards thedrawer. Another variation is to provide a camera moving from one drawerto another. Still another variation is to provide a moving minor foreach drawer and one or more cameras moving between drawers. Themovements of the cameras and mirrors are synchronized to form images ofeach storage drawer. The cameras and drawers may be driven by motors orany means that provide power.

If the image sensing device requires illumination to obtain acceptableimage quality, illumination devices may be provided. For example, LEDsmay be used to illuminate the image area. It is understood that otherillumination sources may be used. In one embodiment, LEDs are disposedsurrounding the lens or image sensors of the camera and light istransmitted along the same path as the camera view. In an embodimentincluding the use of a light directing device, such as a minor, theemitted light would be directed by the minor towards the drawers. Thetiming and intensity of the illumination is controlled by the sameprocessor that controls the camera and its exposure. In some possibleconfigurations of cameras it may be desirable to implement backgroundsubtraction to enhance the image. Background subtraction is a well knownimage processing technique use to remove undesirable static elementsfrom an image. First an image is captured with illumination off. Then asecond image is captured with illumination on. The final image iscreated by subtracting the illumination off image from the illuminationon image. Image elements that are not significantly enhanced by theillumination are thereby removed from the resulting image.

According to another embodiment, for each access, the image sensingsystem 300 is timed to capture at least two images of drawer 300: atleast one image (initial image) captured before a user has access tostorage locations in drawer 300 and at least one image captured afterthe access is terminated, as discussed earlier. The initial image may betaken at any time before the user has access to the contents or storagelocations in the drawer. In one embodiment, the initial image iscaptured when or after a user requests access to system 300, such as bysliding a keycard, punching in password, inserting a key into a lock,providing authentication information, etc. In another embodiment, theinitial image is captured before or in response to a detection of drawermovement from a close position or the unlock of a locking device ofsystem 300.

The data processing system of system 300 determines an inventorycondition based on the initial image, and assigns the determinedinventory condition as “before access” inventory condition; anddetermine an inventory condition based on the image captured after theaccess is terminated and designated the determined inventory conditionas “after access” inventory condition. A change in the inventorycondition of objects in system 300 may be determined based on acomparison of the “before access” and “after access” inventoryconditions or a comparison of the initial image and the image capturedafter the access is terminated.

Concepts and designs described above may be applicable to other types ofstorage systems, such as a type shown in FIG. 1B, where a single doorcontrols the access to multiple shelves or drawers. In one embodiment,the image sensing device may be timed to capture images of the storagelocations when or after a detected termination of access, such asclosing door 250, locking door 250, signing out, etc. It is understoodthat various types of sensors, such as contact sensors, infraredsensors, may be used to determine when a door is closed. Similar to thediscussions earlier, the image sensing device captures images of thestorage locations, and determine an “after access” inventory conditionbased on the captured image. A change in the inventory condition relatedto the access by comparing an inventory condition of the current accessand that of the last access. According to another embodiment, the imagesensing device is timed to take “before access” images of the storagelocations before a user has access to the storage system. For instance,the cameras may be timed to capture images of the storage locations whenor after a user requests access to the system, after detecting anopening of door 250, after receiving authentication information from auser, etc. The storage system determines a “before access” inventorycondition based on the “before access” image. A change in the inventorycondition may be determined according to a difference between the“before access” and “after access” inventory conditions, or a differencebetween the “before access” and “after access” images.

Networked Storage Systems

Storage systems described in this disclosure may be linked to a remoteserver in an audit center, such that inventory conditions in eachstorage system is timely updated and reported to the server. As shown inFIG. 8, a server 802 is coupled to multiple storage systems 800 via awireless network. Server 802 may include a database server, such as aMicrosoft SQL server. Information related to authentication, authorizedusers, inventory conditions, audit trails, etc., is stored in thedatabase.

In one embodiment, each storage system 800 is provided with a datatransceiver, such as an 802.11g or Ethernet module. The Ethernet moduleconnects directly to the network, while a 802.11g module may connectthrough a 802.11g router connected to the network. Each of these networkmodules will be assigned a static or dynamic IP address. In oneembodiment, storage systems 800 check in to the server through the datatransceivers on a periodic basis, to download information aboutauthorized users, authorization levels of different users or differentkeycards, related storage systems, etc. Storage systems 800 also uploadinformation related to the systems such as inventory conditions, drawerimages, tool usage, access records, information of user accessingstorage systems 800, etc., to server 802. Each storage system 800 may bepowered by an AC source or by a battery pack. An uninterruptible powersupply (UPS) system may be provided to supply power during a powerfailure.

Server 802 allows a manager or auditor to review access informationrelated to each storage system 800, such as inventory conditions andinformation related to each access to storage system 800 like userinformation, usage duration, inventory conditions, changes in inventoryconditions, images of drawers or contents of the storage system, etc. Inone embodiment, server 802 may form a real time connection with astorage system 800 and download information from that storage system.The manager or auditor may also program the access control device oneach storage system through server 802, such as changing password,authorized personnel, adding or deleting authorized users for eachstorage system, etc. Authorization data needed for granting access toeach storage system 800 may be programmed and updated by server 802 anddownloaded to each storage system 800. Authorization data may includepasswords, authorized personnel, adding or deleting authorized users foreach storage system, user validation or authentication algorithm, publickey for encryptions and/or decryptions, black list of users, white listof users, etc. Other data updates may be transmitted to each storagesystem from server 802, such as software updates, etc. Similarly, anychanges performed on storage system 800, such as changing password,adding or deleting authorized users, etc., will be updated to server802.

For each access request submitted by a user, a storage systemauthenticates or validates the user by determining a user authorizationaccording to user information input by the user via the data inputdevice and the authorization data. According to a result of theauthentication, the data processor selectively grants access to thestorage system by controlling an access control device, such as a lock,to grant access to the storage system 800 or one or more storage drawersof one or more storage systems 800.

Server 802 also allows a manager to program multiple storage systems 800within a designated group 850 at the same time. The manager may selectwhich specific storage systems should be in group 850. Once a user isauthorized access to group 850, the user has access to all storagesystems within group 850. For instance, a group of storage systemsstoring tools for performing automotive service may be designated as anautomotive tool group, while another group of storage systems storingtools for performing electrical work may be designated as an electricaltool group. Any settings, adjustments or programming made by Server 802in connection with a group automatically apply to all tool storagesystems in that group. For instance, server 802 may program the toolstorage systems by allowing an automotive technician to access all toolstorage systems in the automotive tool group, but not those in theelectrical tool group. In one embodiment, each system 800 only includesminimal intelligence sufficient for operation. All other dataprocessing, user authentication, image processing, etc., are performedby server 802.

Similarly, server 802 also allows a manager to program multiple storagedrawers 330 within a designated group at the same time. The manager mayselect which specific storage drawers, of the same system or differentstorage systems, should be in the group. Once a user is authorizedaccess to the group, the user has access to all storage drawers withinthe group. For instance, a group of storage systems storing tools forperforming automotive tools may be designated as an automotive toolgroup, while another group of storage systems storing tools forperforming electrical work may be designated as an electrical toolgroup.

In another embodiment, an exemplary networked storage system as shown inFIG. 8 utilizes hierarchical authorization architecture to manage accessto storage systems. One or more storage systems 800 are given the statusof master storage system. Each master storage system has one or moreassociated slave storage systems. If a user is authorized to access to amaster storage system, the same user is automatically authorized toaccess any slave storage system associated with that master system. Onthe other hand, if a user is authorized to access a slave storagesystem, the authorization to the slave system does not automaticallygrant the user access to its associated master storage system or otherslave storage systems associated with the same master storage system.

According to still another embodiment, an exemplary networked storagesystem as shown in FIG. 8 grants user access by utilizing multiplehierarchical authorization levels. Each authorization level isassociated with pre-specified storage systems, which can be programmedby a manager via server 802. When a user is assigned a specificauthorization level, this user is authorized to access all storagesystems associated with the assigned authorization level and all storagesystems associated with all authorization levels lower than the assignedauthorization level in the authorization hierarchy, but not to thoseassociated with authorization levels higher than the assignedauthorization level in the authorization hierarchy.

Audit

An exemplary inventory control system according to this disclosuretracks various types of information related to each access. For example,system 800 records date, time and/or duration for each access, andcorresponding user information submitted by a user to obtain access tosystem 800. As discussed earlier, system 800 captures one or more imagesof the storage unit during each access for determining an inventorycondition. The images are linked to each access and accessing user andstored in system 800. System 800 may store the information locally orupload the obtained information to server 802 via the wirelesscommunication network, as shown in FIG. 8.

Server 802 may process and compile the information received from eachsystem 800, to create an audit trail for each server 802. The audittrail is accessible by managers or users with suitable authorizationlevels. Different types of audit trails may be generated and retrievedbased on preference of authorized users. For instance, an audit trailmay be generated for one or more specific dates, one or more specificusers, one or more specific tools, one or more IDs, etc. Additionalinformation and analysis may be generated and provided by server 802.For example, system 802 may track usages of a specific tool over time,and generate a report summarizing a usage frequency for each tool forevaluation. Such report may be used to determine what tools are usedmore frequently and which tools probably are not needed because they areused less often than others.

FIG. 9 a shows an exemplary screen of an audit trail with respect to aspecific storage system 800. Each access to system 800 is identified byDate/Time 920 and user information 910 of users associated with eachaccess. User information may include any information submitted by a userwhen requesting access to system 800, such as finger prints, facialrecognition images, user images taken by user cameras, passwords,information stored in keycards, any information for authentication, etc.In one embodiment, data of user facial characteristics of each user isstored in system 800 or server 802. For each access, an image of a useraccessing system 800 is captured by a user camera. User informationsubmitted by the user for gaining access to system 800, such asinformation stored in a keycard and/or password, is collected. Thecaptured image is compared against user facial characteristics of a useridentified by the user information. System 800 or server 802 determineswhether the facial characteristics of the user accessing system 800matches facial characteristics of the user identified by the userinformation.

One or more images are taken during each access to storage system 800.FIG. 9 b shows an exemplary “before access” image taken by cameras ofsystem 800 before a user has access to the storage locations or when thedrawer is moving in the first direction, as discussed earlier in thisdisclosure. As shown in FIG. 9 b, each tool is properly stored in itscorresponding storage location. FIG. 9 c shows an exemplary “afteraccess” image taken by cameras of system 800 after the access isterminated or when a storage drawer moves in the second direction asdiscussed earlier. As shown in FIG. 9 c, tools corresponding to storagelocations 951 and 952 are missing. Based on the image shown in FIG. 9 c,system 800 determines that tools in storage locations 951 and 952 aremissing. An audit trail is generated regarding the missing tools and theuser associated with the access. FIG. 9 d shows an exemplary recordstored in system 800 and/or server 802, in which both “before access”and “after access” images 981 and 982 are stored. Missing tools areidentified according to “after access” image 982 and listed in area 980.

In the previous descriptions, numerous specific details are set forth,such as specific materials, structures, processes, etc., in order toprovide a thorough understanding of the present disclosure. However, asone having ordinary skill in the art would recognize, the presentdisclosure can be practiced without resorting to the detailsspecifically set forth. In other instances, well known processingstructures have not been described in detail in order not tounnecessarily obscure the present disclosure.

Only the illustrative embodiments of the disclosure and examples oftheir versatility are shown and described in the present disclosure. Itis to be understood that the disclosure is capable of use in variousother combinations and environments and is capable of changes ormodifications within the scope of the inventive concept as expressedherein.

1. An inventory control system for monitoring the removal andreplacement of objects, the system comprising: at least one storagedrawer, each storage drawer including at least one storage location forstoring objects, wherein each storage drawer is configured to move in afirst direction allowing increasing access to storage locations of thedrawer, and a second direction allowing decreasing access to storagelocations of the drawer; at least one image sensing device configured toform at least one first image of the storage locations when a respectivestorage drawer moves in the second direction or when the respectivestorage drawer stops moving in the second direction; a data processorconfigured to receive information representing the at least one firstimage of the storage locations, and determine an inventory condition ofthe objects stored in the respective storage drawer according to the atleast one first image. 2-56. (canceled)